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Abstract 

Objectives: This consensus conference presentation article focuses on methods of measuring crowding. 
The authors compare daily versus hourly measures, static versus dynamic measures, and the use of linear or 
logistic regression models versus survival analysis models to estimate the effect of crowding on an outcome. 

Methods: Emergency department (ED) visit data were used to measure crowding and completion of wait- 
ing room time, treatment time, and boarding time for all patients treated and released or admitted to a single 
ED during 2010 (excluding patients who left without being seen). Crowding was characterized according to 
total ED census. First, total ED census on a daily and hourly basis throughout the 1-year study period was 
measured, and the ratios of daily and hourly census to the ED's median daily and hourly census were com- 
puted. Second, the person-based ED visit data set was transposed to person-period data. Multiple records 
per patient were created, whereby each record represented a consecutive 15-minute interval during each 
patient's ED length of stay (LOS). The variation in crowding measured statically (i.e., crowding at arrival or 
mean crowding throughout the shift in which the patient arrived) or dynamically (every 15 minutes 
throughout each patient's ED LOS) were compared. Within each phase of care, the authors divided each 
individual crowding value by the median crowding value of all 15-minute intervals to create a time-varying 
ED census ratio. For the two static measures, the ratio between each patient's ED census at arrival and the 
overall median ED census at arrival was computed, as well as the ratio between the mean shift ED census 
(based on the shift in which the patient arrived) and the study ED's overall mean shift ED census. Finally, the 
effect of crowding on the probability of completing different phases of emergency care was compared when 
estimated using a log-linear regression model versus a discrete time survival analysis model. 

Results: During the 1-year study period, for 9% of the hours, total ED census was at least 50% greater 
than the median hourly census (median, 36). In contrast, on none of the days was total ED census at 
least 50% greater than the median daily census (median, 161). ED census at arrival and time-varying ED 
census yielded greater variation in crowding exposure compared to mean shift census for all three 
phases of emergency care. When estimating the effect of crowding on the completion of care, the dis- 
crete time survival analysis model fit the observed data better than the log-linear regression models. The 
discrete time survival analysis model also determined that the effect of crowding on care completion 
varied during patients' ED LOS. 

Conclusions: Crowding measured at the daily level will mask much of the variation in crowding that 
occurs within a 24-hour period. ED census at arrival demonstrated similar variation in crowding expo- 
sure as time-varying ED census. Discrete time survival analysis is a more appropriate approach for 
estimating the effect of crowding on an outcome. 
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STATUS OF MEASURING CROWDING IN THE ED 

During the past decade, advancements have been 
made in the measurement of crowding in the 
emergency department (ED). Historically, 
crowding was most often measured by the frequency of 
ambulance diversion or by provider opinion surveys. 1-3 
However, these measures were not highly reliable 
because EDs use different criteria to trigger ambulance 
diversion, and provider opinion questions were not 
standardized or well validated. With the uptake of ED 
information systems (EDIS), and the appreciation that 
crowding needs to be measured in a more objective and 
reliable way, numerous quantitative measures have been 
proposed. 4 The majority of these recent measures char- 
acterize crowding in terms of an input (e.g., number of 
arrivals), throughput (e.g., number of patients being trea- 
ted), or output (e.g., boarding time) factor. 4,5 Some 
crowding measures are multidimensional, such as the 
Emergency Department Work Index (EDWIN) or the 
National Emergency Department Overcrowding Scale 
(NEDOCS). 6,7 Rarely have the different crowding mea- 
sures been directly compared. The few comparisons 
performed suggest that the measures may be inter- 
changeable; they yield similar conclusions regarding the 
effect of crowding on different outcomes. 8-11 However, 
one meaningful difference is that some measures are 
more feasible for routine use because they require fewer 
data elements, and the data elements required are com- 
monly captured electronically even in facilities that do 
not have a comprehensive EDIS. 8 

Despite the number of crowding measures developed 
in recent years, little attention has been paid to the 
appropriate methods of measuring crowding and quan- 
tifying the effects of crowding on ED length of stay 
(LOS). Should crowding be measured on a daily or 
hourly basis? Should crowding be measured statically, 
such as using crowding at the time of a patient's arri- 
val, or pooled over a period of time? Or should it be 
measured dynamically, acknowledging that the degree 
of crowding varies throughout a patient's ED stay? 
Does the effect of crowding on an outcome change 
over time? This presentation from the June 2011 Aca- 
demic Emergency Medicine consensus conference 
"Interventions to Assure Quality in the Emergency 
Department" compares methods of measuring 
crowding and provides insight into the strengths and 
limitations of different approaches. 

Characteristics of the Study Site (Table 1) 

To compare crowding methods, we used 2010 ED visit 
data from a community teaching hospital affiliated with 
a major academic center. The hospital has an acute care 
inpatient bed capacity of 318 licensed and staffed beds. 
The study ED consists of 31 treatment spaces in the 
main ED and an additional eight spaces in an adjacent 
urgent care area. There is a remote observation unit 
that is staffed by inpatient physicians. Both adult and 
pediatric patients are treated at the study ED. 

The ED volume in 2010 was 58,517 and the mean 
(±SD) LOS was 6.3 (±5.4) hours for discharged patients 
and 8.9 (±5.5) hours for admitted patients. The study ED 
is primarily staffed by physicians and physician 



Table 1 

Description of Study Site for January 1, 2010-December 31, 2010 



ED Characteristics 


Annual ED volume 


58,517 


Annual median admission rate 


22.5% 


Annual median left without being 


6.0% 


seen rate 




ED treatment capacity 


39 


Main ED treatment spaces 


31 


Fast track treatment spaces 


8 (open daily 




between 8AM-2AM) 


Median service completion times, minutes (IQR) 


Waiting room time 


24 (11-64) 


Treatment time 


203 (120-341) 


Boarding time 


108 (52-205) 


Crowding measured by total ED census 




Low crowding (0-24th percentile) 


6-27 


Medium crowding (25th-74th 


28-46 


percentile) 




High crowding (>75th percentile) 


47-90 


Median staffing per hour (IQR) 




Doctors 


3 (2-3) 


Fellows/residents/physician 


4 (2-5) 


extenders 




Nurses 


13 (13-14) 


Technicians 


4 (3-4) 


IQR = interquartile range. 



assistants. Residents from the affiliated academic teach- 
ing hospital work in the study ED, but are not there 
24/7. The ED uses MEDITECH Client Server Version 
5.54 (Westwood, MA) for all patient care and tracking 
activities except physician documentation. The primary 
providers document using paper template charts, which 
are scanned and stored electronically in the EDIS. 
MEDITECH interfaces with all of the hospital's other 
electronic clinical information systems and is also the 
inpatient information system. 

From MEDITECH, we downloaded the following data 
elements for all ED visits that occurred between Janu- 
ary 1, 2010, and December 31, 2010: 1) demographic 
characteristics (age and sex), 2) clinical characteristics 
(chief complaint, triage level, and mode of arrival), 3) 
date and time of arrival, 4) date and time of room place- 
ment, 5) date and time of decision to admit to an inpa- 
tient or observation bed, 6) date and time of 
disposition, and 7) disposition status. The study ED uses 
the Emergency Severity Index to triage patients. 12 The 
triage nurse documents each patient's chief complaint 
using a standardized list in MEDITECH. 

MEDITECH has a patient tracking system that auto- 
matically records the time when a staff member moves 
the patient from one status event to another. For exam- 
ple, when the patient is placed in a room, the treating 
nurse changes the status from "waiting for treatment 
room" to ''in room." The only activity status that is not 
captured by the tracker is the admission decision time. 
A member of the registration staff manually enters the 
admission decision time according to documentation 
provided by the attending physician on the admission 
request form for an inpatient or observation bed. In a 
previous study that used the same data elements, we 
examined the accuracy of the electronic patient tracking 
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data for a small random sample of 275 ED visits and 
found that the electronic data accurately captured 
patients' movement from one phase of care to the 
next. 13 

Based on the data, we calculated each patient's wait- 
ing room time, treatment time, and boarding time, if 
admitted. Waiting room time was defined as the 
interval between arrival time and time of room place- 
ment. Treatment time was defined as the interval 
between room placement and physical departure from 
the ED for discharged patients. For admitted patients, 
treatment time was the interval between room 
placement and the admission decision time. Boarding 
time was defined as the period between the admission 
decision time and physical departure from the ED. 

Crowding was measured using ED census. We calcu- 
lated ED census as the total number of patients physi- 
cally in the ED during a specified time interval. Using 
the total ED census distribution, we categorized crowd- 
ing as low (<25th percentile), medium (25th-75th per- 
centiles), and high (>75th percentile). This study was 
approved by the institutional review boards of both 
institutions as part of a career development grant 
awarded to the first author. 

More Variation in Within-day Crowding Than 
Between-day Crowding 

When measuring crowding, one of the first questions 
that must be addressed is how frequently should 
crowding be measured? If one measures crowding on a 
daily basis, is this sufficient to accurately capture the 
effect that crowding has on the outcomes of patients 
treated in the ED that day? In fact, there is more varia- 
tion in crowding within a 24-hour period than across 
24-hour periods. Crowding can double within a single 
day but rarely doubles from one day to another. To 
illustrate this point, we measured total ED census 
hourly for the 1-year study period (n = 8,760 hours). 
The median ED census for all 8,760 hours was 36. We 
divided the ED census of each study hour during the 
1-year period by the median hourly ED census to create 
an hourly ED census ratio. The advantage of standard- 
izing the individual values to the median is that the 
ratio has the same interpretation when measuring 
crowding across multiple EDs. When this ratio is equal 
to 1, it means that the ED census for a particular hour 
is equal to the median hourly ED census. When it is 
lower than 1, the hourly ED census is lower than the 
median, and when it is greater than 1, it is higher than 
the median. The top histogram in Figure 1 shows the 
percentage of hours that the ED census was various 
distances above or below the median. The variance of 
this hourly census ratio represented in the top 
histogram is 0.12. 

We used this same approach to characterize the 
variation in crowding on a daily basis. We measured 
total ED census on a daily basis for the 365 days of 
the study period and divided daily ED census by the 
median daily ED census. The median daily census for 
the study ED was 161. The bottom histogram in 
Figure 1 shows the distribution of daily values relative 
to the median. As expected, Figure 1 illustrates that 
variation in crowding is larger hourly than daily. 
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Figure 1. (Top) Hourly ED census divided by the median 
hourly census for 8,760 hours during 2010. (Bottom) Daily ED 
census divided by the median daily census for 365 days in 
2009. The top histogram indicates there is greater variation in 
hourly ED census than daily ED census at the study ED. 

During the 1-year period, 9% of the hours had a total 
ED census at least 50% greater than the median hourly 
census. In contrast, for none of the days was the total 
ED census at least 50% greater than the median daily 
census. The variance of this distribution of daily ED 
census ratio is 0.01, 1/12 as large as the hourly 
distribution. As variance is a measure of information, 
the hourly variation has the potential to provide 12 
times as much information, for example, about the 
effects of crowding on LOS or patient outcomes. The 
results of Figure 1 demonstrate that much of the 
variation in crowding will be undetected by daily 
measures. 

Measuring Crowding Statically or Dynamically 

A second important question is: how accurate is a sin- 
gle measure of crowding, whether it be at one time, 
or averaged over time, compared to measuring crowd- 
ing multiple times throughout each patient's ED stay? 
Figure 2 illustrates the total ED census measured 
hourly for 31 days in August 2010 at the study ED. On 
August 23, the line bolded in black shows that if a 
patient arrives at 8:00am and crowding is measured at 
the time of arrival, then the patient's crowding value 
would be 22. If this patient was discharged at noon 
and crowding is measured by the average ED census 
during that patient's 4-hour ED LOS, the crowding 
value would be 31. If crowding is measured by the 
mean census during the shift in which the patient 
arrived (8am-4pm), the crowding value (shift mean ED 
census) would be 42. 

To compare the variation in crowding that is mea- 
sured at a single time point or pooled over an interval, 
we measured total ED census every 15 minutes during 
the 1-year study period and compared these frequent 
measures of crowding to crowding measured at arrival 
as well as crowding averaged over the shift for 54,706 
patients in the dataset (excluding those who left without 
being seen). 
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Figure 2. Observed hourly ED census at the study ED for each day in August 2010. August 23 is highlighted by a thick black line to 
illustrate the difference in ED census if measured at arrival or averaged across a patient's LOS or across the shift in which the 
patient arrived. LOS = length of stay. 



To do this, we transposed the ED visit data from a 
person-based file to a person-period file. We divided 
each patient's LOS into 15-minute intervals and for 
each 15-minute interval that the patient remained in 
the ED, we created a period-specific record for that 
person and interval. Thus, the person-period file has 
multiple records per subject, one for each time period. 
Table 2 illustrates the conversion of the data set from 
person only to person-period. In the person-based file, 
the first subject had a total ED LOS of 133 minutes, so 
we created nine person-period records for this subject, 
one for each consecutive 15-minute interval. The per- 
son-based record shows that crowding, measured 
every 15 minutes (C1-C9) ranged from 21 at arrival to 
60 during the first subject's ED LOS. In the person-per- 
iod file, we used dummy variables (D1-D9) to indicate 
each 15-minute time period and to match the values of 
the covariates to the appropriate time period. For 
example, crowding at arrival (CI) is recorded in the 
first time interval record (Dl). Likewise, the time that 
each subject completed a stage of care is indicated by 
a 1 in the interval in which it occurred. For the first 
subject, completion of waiting room time is indicated 
in the D4 record, decision to admit is recorded in the 
D6 interval, and ED discharge is documented in the D9 
record. 

Within each phase of care, we divided each individ- 
ual crowding value by the median crowding value of all 
15-minute intervals to create a time-varying ED census 
ratio. For the two static measures, we computed the 
ratio between each patient's ED census at arrival and 



the overall median ED census at arrival, as well as the 
ratio between the mean shift ED census (based on the 
shift in which the patient arrived) and the study ED's 
overall average shift ED census. 

Figure 3 displays nine histograms of the distribution 
of the time-varying ED census ratio; ED census at arri- 
val ratio; and the shift mean ED census ratio for waiting 
room time, treatment time, and boarding time. For all 
three phases of care (and for overall LOS), the 
time-dependent census and ED census at arrival show 
greater distances from the median compared to ED 
census averaged over the shift. For waiting room time, 
the variance is largest for ED census at arrival (0.10) 
compared to time-varying ED census (0.08) or mean 
shift census (0.07). In contrast, for treatment time and 
boarding time, the variance is approximately the same 
for time-varying ED census and ED census at arrival 
(0.10 and 0.09, respectively) and smaller for mean 
census during the shift (0.07). 

In this comparison, ED census at arrival demon- 
strated similar variation in crowding as the 
time-varying crowding measure. This finding was 
unexpected and may be partially due to the fact that 
ED census does not vary as much by hour of day as 
compared to another crowding measure, such as 
hourly ED arrivals. 14 The more hourly variation in the 
crowding measure, the more likely we would have 
seen greater differences between the dynamic and 
static measures. Despite the small differences between 
time-varying ED census and ED census at arrival, we 
recommend using the dynamic measure if possible 
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Table 2 

Conversion from Person-based to a Person-period Data Set 
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Figure 3. (Top row) ED census every 15 minutes divided by the median 15-minute interval census for each patient during the three 
phases of emergency care. (Middle row) ED census at arrival divided by the median ED census at arrival for each patient during 
the three phases of emergency care. (Bottom row) Mean shift ED census divided by the median mean shift ED census for each 
patient during the three phases of emergency care. 
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because it measures the crowding exposure at differ- 
ent points in time during a patient's LOS. While 
crowding may have a negative effect on all patients in 
the ED, it will likely affect some patients more than 
others depending on where patients are in their treat- 
ment trajectory and how crowded it is at the moments 
in time when they are competing for different 
resources. 

Estimating the Effect of Crowding Using Linear or 
Logistic Regression Models 

The vast majority of studies that have documented the 
effect of crowding on patient care and outcomes have 
measured crowding using linear or logistic regression 
models. 15,16 Below is an equation for a linear regression 
model that shows the expected mean value (E) of an 
outcome (Y) given a set of predictors (X). The major dif- 
ferences between linear and logistic regression models 
are: 1) in linear regression, the expected value of the 
conditional mean can take on any value as x ranges 
from negative to positive infinity, whereas the expected 
value of the mean is bounded between 0 and 1 with a 
dichotomous outcome; and 2) the error term (s) in logis- 
tic regression follows a binomial distribution rather 
than a normal distribution, as in linear regression. 17 
The simple equation below can be used to predict mean 
waiting room time (log-transformed since waiting room 
time is skewed) as a function of crowding, patient, and 
clinical characteristics: 

E(Y\X) = p 0 + p t (Crowding) + p 2 (Age) + p 3 (Sex) 
+ p 4 (Acuity) + p 5 (Chief complaint) 

Y = E(Y|X) + £,£~N(0,<7 2 ) 

Linear and logistic regression models that include 
crowding as a predictor are constrained in two impor- 
tant ways: 1) the exposure of crowding is represented 
by one measurement of crowding at a single time point 
or pooled across an interval as discussed earlier and 
2) the model assumes that the effect of crowding on the 
outcome does not change over the course of time, such 
as during a patient's LOS. In a linear or logistic regres- 
sion model, there is usually only one parameter that 
estimates the effect of crowding on the outcome. 

Estimating the Effect of Crowding Using a Discrete 
Time Survival Analysis Model 

Another approach to estimating the effect of crowding 
on a dichotomous or continuous outcome is with a dis- 
crete time survival analysis model. 18 This model is 
advantageous when an investigator is interested in 
determining not only if an outcome occurs, but when it 
occurs. The discrete time survival analysis model esti- 
mates the discrete time hazard (h t ) or conditional prob- 
ability that an individual will experience an outcome (Y) 
in time period t given that the individual did not 
experience the outcome in a prior time period: 
h t = Pr[Y=t\Y>t\. 



We can plot the discrete time probability parameters 
h t against time (hazard function) as the population risk 
of the outcome occurring in time period t, conditional 
on the outcome not having occurred previously. This 
hazard function plot describes how the risk of an out- 
come occurring changes over time. In addition to esti- 
mating the baseline risk of an outcome occurring, the 
discrete time survival analysis model can also estimate 
how a risk profile changes by the presence of different 
predictors. To estimate the shift in baseline risk associ- 
ated with a predictor, the hazard probabilities are repa- 
rameterized so that the model estimates the log odds of 
the outcome occurring as a function of different predic- 
tors. An example of a simple discrete time survival 
analysis model that estimates the odds of being placed 
in a treatment room (h) for a given individual (i) in a 
specific time period (t) for different crowding levels is 

Logit e (h it ) = [otiDiit + oe 2 D 2it + . . . + ot 9 D 9if ] 
+ p lit (Crowding t ) + p 2 (Age) 
+ p 3 (Sex) + p 4 (Acuity) 
+ p 5 (Chief complaint) 

In a discrete time survival analysis model, time (t) is 
a predictor and is represented by multiple intercepts 
(Di-Dg), one for each time period. When all predictors 
are equal to 0, these a intercepts estimate the odds of 
the outcome occurring (i.e., completion of treatment) 
over time. The /? coefficients represent the relative odds 
associated with different predictor variables. The model 
above includes one time-varying predictor and four 
time-invariant predictors. Time-varying or time-depen- 
dent predictors are those whose values can change 
over time, such as crowding, staffing, availability of 
resources, etc. Time-invariant predictors are factors 
that remain constant throughout the study period, such 
as sex, acuity, and chief complaint. 

The above model can be extended to include an inter- 
action term of time x crowding to determine whether 
the effect of crowding on the odds of completing differ- 
ent phases of emergency care changes over time. Most 
studies to date have examined whether crowding 
affects the occurrence of an event within a given time 
period, such as within 4 hours of arrival or within 
7 days of ED discharge. However, it is likely that the 
effect of crowding on the occurrence of an outcome 
varies over time (for example, greater earlier in the time 
period and less influential later on or vice versa). The 
discrete time survival analysis model allows one to eval- 
uate whether the effect of crowding on an outcome is 
constant or varies over time. 

Using the person-period data set we described in the 
previous section, Figure 4 illustrates that the impact of 
crowding on the odds of completing care changes over 
time. If an interaction term of crowding and time is not 
included in the discrete time survival analysis model, 
then the risk of completing a phase of care is assumed 
to remain the same throughout a patient's stay and is 
represented by the dashed lines in each of the graphs. 
However, when an interaction term is included, it is 
clear that the effect of crowding on completion of care 
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Figure 4. Relative odds of completing each phase of emergency care depends on whether the effect of crowding on completion of 
care is allowed to vary over time. The dashed line displays the relative odds of completing care when the ED is highly crowded 
(upper quartile) versus not (lowest quartile), assuming that the odds are constant regardless of how long a patient is in a particular 
phase of care. The solid line shows how the relative odds (gray band is the 95% confidence interval) of completing care by different 
crowding levels changes over the duration of a phase of care. 



changes over time. For example, in the waiting room 
time model, the relative odds of completing waiting 
room time during highly crowded periods versus low 
crowding periods is highest at arrival and decreases 
the longer the patient is in the ED. In contrast, for 
boarding, the odds of completing boarding is initially 
lower for patients who are in the ED during high ver- 
sus low crowding periods, but increases steadily the 
longer they wait for an inpatient bed. 

To directly compare crowding results estimated from 
a discrete time survival analysis model to those from a 
linear or logistic regression model is difficult because 
of the fundamental differences among the models. The 
discrete time survival analysis model can estimate the 
effect of crowding dynamically, whereas the linear and 
logistic regression cannot. However, they can both esti- 
mate the hazard function or the conditional probability 
of completing a phase of care at each time interval by 
different crowding levels. In the linear regression 
model, we log-transformed each outcome (i.e., waiting 
room time, treatment time, and boarding time) and 
used the normal assumption on the log scale, which 
corresponds to a log-normal distribution to estimate 
the service completion times. For the discrete time sur- 
vival analysis model, the hazard function is estimated 
from the conditional probabilities of completing care in 
each time interval. 

In Figure 5, we have plotted the observed and pre- 
dicted hazard of completing waiting room time, treat- 
ment time, and boarding time by the different models 
(i.e., discrete time survival analysis vs. linear regression) 



and crowding levels. Crowding has the biggest effect 
on waiting room time, and patients who are in the ED 
during high crowding periods have a lower probability 
of being placed in a room compared to those in the ED 
during low crowding periods, regardless of the model 
used. In addition, the predicted probabilities estimated 
from the discrete time survival analysis model fit the 
observed data better than the estimates from the linear 
regression models that measure crowding either at time 
of arrival or as a shift average. The discrete time sur- 
vival analysis model is more flexible to describe the 
actual hazard rather than imposing a parametric 
assumption such as the lognormal distribution. It also 
allows for the inclusion of an interaction term of 
crowding and time compared to the linear regression 
model, which does not because it cannot accommodate 
dynamic predictors. The linear regression models are 
frequently underestimating the probability of comple- 
tion of care. For waiting room time, this only occurs 
during low crowding periods. For treatment time and 
boarding time, the linear regression models initially 
overestimate the rate of completion of care for the first 
1 to 2 hours and then underestimate it thereafter. 

CONCLUSIONS 

The methods used to measure crowding will affect the 
inferences one makes about the relationship between 
crowding and an outcome. First, the more granular 
the measurements of crowding, the more information 
is preserved to study crowding effects on health 



1276 



McCarthy et al. • METHODS OF MEASURING CROWDING 



Low Crowding 



Median Crowding 



High Crowding 




i i i i i r 
0 1 2 3 4 5 

Follow-up Time (hr) 



1 S 





Follow-up Time (hr) 



i i r 
12 3 4 

Follow-up Time 



(hr) 



Figure 5. Observed and predicted probability of completing different phases of emergency care based on whether there is low, 
medium, or high crowding. The gray line is the observed probability of completing a phase of care over each patient's entire phase 
of care. The solid black line estimates the predicted probability of completing care based on the discrete time survival analysis 
model. The dashed black line estimates the probability of completing care based on crowding measured at time of ED arrival. The 
black dotted line estimates the probability of completing care based on crowding averaged over the shift in which each patient 
arrived. 



outcomes. In particular, crowding that is measured at 
the daily level will mask much of the variation in crowd- 
ing that occurs within a 24-hour period. Second, of the 
two static measures, ED census at arrival demonstrates 
more variation in crowding compared to mean ED 
census during the shift. ED census at arrival and time- 
varying ED census demonstrated similar variation in 
crowding exposure on ED LOS. Finally, the discrete 
time survival analysis model is a preferred, more flexible 
approach to estimating the effect of crowding on an out- 
come because it allows one to include time-varying pre- 
dictors and to evaluate whether the influence of 
different predictors on an outcome is constant or 
changes over time. 
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